Skip to content

feat: add SQLite database persistence for game recording#8

Merged
mdgoldberg merged 22 commits intomainfrom
feature/database
Feb 2, 2026
Merged

feat: add SQLite database persistence for game recording#8
mdgoldberg merged 22 commits intomainfrom
feature/database

Conversation

@mdgoldberg
Copy link
Copy Markdown
Owner

@mdgoldberg mdgoldberg commented Jan 19, 2026

Summary

Adds a new database crate that enables SQLite persistence for recording games, players, and actions. Integrates with the simulation crate to automatically track all game activity.

Changes

New Crate: database

  • Models: PlayerRecord, GameRecord, GameResultRecord, ActionRecord, FailedWrite
  • Writers: DatabaseWriter trait with BulkGameWriter and StreamingGameWriter implementations (async sqlx)
  • Config: Database URL from CLI, env, or YAML config; writer type selection (bulk/streaming)
  • Retry Logic: Exponential backoff for write operations
  • Collectors: GameEventCollector and GameMetadata for data collection

Integration

  • Simulation crate now records all game events to database via DatabaseWriter trait
  • Player registration with UUID-based identification
  • Configurable database URL (sqlite::memory: for testing, sqlite:file.db for persistence)
  • Choice of bulk (atomic) or streaming (real-time) persistence modes

Documentation

  • Added comprehensive README with usage examples
  • Added config.sample.yaml with 5-player example
  • Added AGENTS.md docs for simulation and types crates
  • Added database/README.md with architecture details

Tests

  • Added integration tests for game execution (simulation/tests/)
  • Added unit tests for database operations (database/src/tests/)
  • Added database integration tests (database/tests/)

Breaking Changes

  • GameState::new now accepts (Uuid, name, strategy) tuples instead of (name, strategy)

Usage

# In-memory (no persistence)
cargo run --bin run_simulation -- --config config.yaml

# Persistent SQLite database
cargo run --bin run_simulation -- --config config.yaml --database sqlite:roosevelt.db

Matt Goldberg added 9 commits January 19, 2026 03:13
- Add trailing slash to target/ for consistency
- Add config.yaml to ignore user configs
- Add sqlite* to ignore SQLite database files
- Add .sqlx/ to ignore sqlx macro cache directory
- Add full project overview explaining Roosevelt card game simulation
- Document quick start with prerequisites and installation steps
- Add YAML configuration examples for various player setups
- Document database persistence features and configuration priority
- Explain game rules, roles, and card passing mechanics
- List available AI strategies (default, random, input)
- Document command-line options and database schema
- Update AGENTS.md to reflect project structure changes
- Add database crate as new workspace member
- Define shared workspace dependencies for all crates
- Include sqlx, r2d2, tokio, async-trait, and other DB-related deps
- Update Cargo.lock with all transitive dependencies
Add new database crate using sqlx with SQLite support for persisting:
- Players: unique IDs, names, and creation timestamps
- Games: start/end times, deck seeds, player order, config snapshots
- Game results: finishing places and role assignments
- Actions: all game actions with turn order and phase tracking
- Failed writes: error tracking for debugging

Features:
- Connection pooling with r2d2 (max 20 connections)
- Retry logic with exponential backoff for writes
- Async trait for game recording operations
- Migration-based schema management
Add config.sample.yaml with 5-player example configuration showing:
- Player names and strategy selections
- Delay configuration for visualization
- Database URL configuration
Update simulation crate to use database persistence:
- Add database dependency to Cargo.toml
- Modify run_game to accept GameRecorder trait for recording games
- Record all game actions (plays, sends, passes) with turn order and phase
- Record game results with finishing places and roles
- Add player registration flow with database lookup
- Support configurable database URL via CLI, env, or YAML config
Changes to support UUID-based player identification:
- CardPlay::Ord now compares by size first (Single < Pair < Triple < Quad)
- GameState::new accepts (Uuid, name, strategy) tuples for player inputs
- Fix inverted is_first_cardplay logic (was checking all instead of any)
- Add PlayerState::new_with_id constructor with explicit UUID
- Export Hand trait from lib.rs for hand detection utilities
Add integration tests for simulation crate:
- test_run_game_with_default_strategies: 3-player game with default AI
- test_run_game_with_mixed_strategies: mix of default and random AI
- test_run_game_with_delay: verify delay functionality works
- test_multiple_games: consecutive games with new player inputs

Add database unit tests:
- test_record_and_retrieve_player
- test_record_game, test_record_action, test_record_game_result
- test_finish_game, test_noop_recorder

Update project documentation:
- simulation/AGENTS.md: Simulation crate overview
- types/AGENTS.md: Types crate documentation

Fix clippy warnings:
- Remove redundant tokio imports
- Rename tests module to database_tests to fix module-inception
The #[tokio::main] attribute already handles tokio imports, so
the explicit  statement is unnecessary.
Add WAL journal mode and normal synchronous mode to SQLite connection options
for improved concurrency and balanced performance. Increase cache size to -64000
pages (approximately 64MB) to enhance performance during bulk write operations,
preventing excessive disk I/O and improving overall database throughput.

This change addresses performance bottlenecks identified in bulk data operations
by leveraging SQLite's WAL mode for concurrent readers/writers and optimizing
memory usage for write-heavy workloads.
Matt Goldberg added 11 commits January 28, 2026 00:11
- Add UuidParsing variant to DatabaseError enum with From<uuid::Error> trait
- Replace unwrap() call in get_player_by_name with proper error propagation
- Fix closure return type to handle Result correctly

Prevents application crashes when database contains malformed UUID strings.
- Replace ROOSEVELT_DATABASE with DATABASE_URL in README documentation
- Aligns with Rust/SQLx ecosystem conventions
- Improves tooling compatibility and developer experience

Follows established patterns used by SQLx, Diesel, and major Rust applications.
Replace the old GameRecorder interface with a new DatabaseWriter trait that supports
bulk and streaming implementations for flexible game recording.

Core Changes:
- Add DatabaseWriter trait with GameHandle-based game tracking
- Implement BulkGameWriter for atomic single-transaction game recording
- Implement StreamingGameWriter for real-time persistence
- Add GameEventCollector for in-memory event collection
- Add GameMetadata for game configuration

Infrastructure:
- Create database/collectors/ module for data collection types
- Create database/writers/ module for writer implementations
- Add GameHandle type (wrapper around i64) for game identification
- Add DatabaseWriterType config enum for writer selection
- Implement FromStr trait for DatabaseWriterType parsing

Integration:
- Update simulation/src/lib.rs to use new DatabaseWriter API
- Update run_simulation.rs to use BulkGameWriter
- Update integration tests for new mutable writer interface
- Keep NoopRecorder for no-persistence scenarios

Cleanup:
- Remove old repository.rs and DatabaseRecorder
- Remove legacy GameRecorder trait
- Clean up lib.rs exports

Bug Fixes:
- Fix migrations path from ./migrations to database/migrations
- Fix BulkGameWriter::record_player to actually insert players
- Resolve foreign key constraint issues

Documentation:
- Create comprehensive database/README.md
- Update AGENTS.md with new database structure
- Update workspace crate count from 3 to 4
- Document writer selection configuration

Testing:
- Add 7 integration tests for DatabaseWriter implementations
- Add unit tests for GameMetadata, GameEventCollector
- Total: 15 tests passing (4 unit + 7 integration + 4 simulation)

Verification: cargo test -p database -p simulation
All 15 tests passing, code compiles cleanly
- Update README.md to reference DatabaseWriter instead of GameRecorder
- Add reference to database/README.md for detailed DatabaseWriter docs
- Update AGENTS.md to reflect test infrastructure now exists
- Remove outdated 'No tests' note from AGENTS.md

Verification:
- grep 'GameRecorder' README.md returns no results
- grep 'DatabaseWriter' README.md returns 2 matches (both updated)
@mdgoldberg mdgoldberg merged commit cc8ee8d into main Feb 2, 2026
4 checks passed
@mdgoldberg mdgoldberg deleted the feature/database branch February 2, 2026 20:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant